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(57) Provided is a personal communications device 
having a touch sensitive display screen, in which text 
may be marked by a user to be used with different appli- 
cations. In particular, to begin marking, a user has to 
maintain his touch at a location on the screen for a short 
predetermined period of time until the system detects 
that the position is held over the same character for a 
delayed period, at which time the system beeps to pro- 
vide an indication that there has been a change of oper- 
ation mode. The marking mode continues in operation 
until the user removes his contact from the touch screen. 
Thus : text over which the contact point passes after the 
initiation of the marking mode is marked, until the user 
removes his contact point from the touch screen. A sec- 
ond beep is sounded to alert the user of the termination 
of the marking mode, and the appearance of a menu of 
operations on the screen. The menu of operations ech- 
oes the marked text for user confirmation. The marked 
text may be used by the user in a number of applications, 
among which include a calling application whereby the 
marked text is used to place a call. 
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Description 

FIELD OF THE INVENTION 

The present invention relates to data processing 
system and more particularly to a personal data process- 
ing and communications system that has a touch sensi- 
tive display screen. 

BACKGROUND OF THE INVENTION 

This invention relates to a co-pending application 
entitled "Field Programming Apparatus and Method for 
Updating Programs in a Personal Communications De- 
vice" assigned to the same assignee of the instant ap- 
plication. The disclosure of the just noted application is 
incorporated by reference herein. 

As disclosed in the just noted application, a personal 
communications device SIMON, announced by the IBM 
Corporation in 1 994, includes many features for facilitat- 
ing personal communications. As shown in Figs. 1 and 
2 : the SIMON personal communications device 2 in- 
cludes the facilities of a cellular telephone and of an elec- 
tronic pager. In addition, it is capable of facsimile trans- 
mission and reception, as well as electronic mail sending 
and receiving. Moreover, a computer note pad, address 
book and calendar are all provided within the system 2. 

As shown, system 2 has a speaker 4, an on and off 
switch 6, volume adjust switches 8a and 8b, a touch sen- 
sitive screen 10, a PCMCIA card receptacle slot 12, a 
PCMCIA card release switch 14, a battery pack 16, an 
input/output connector 18 and a microphone 20. Also 
provided with the system is a stylus 22, which may be 
used to interact with touch sensitive screen 10. It should 
be noted, however, that in place of stylus 22, a user can 
also use his finger. To act as a cellular device, an antenna 
24 is extendable from the body of the device 2. 

Inasmuch as data is stored in the SIMON device (for 
example in the address book) and the device itself is ca- 
pable of receiving different types of messages via fax 
and E-mail messages, to be useful, it becomes incum- 
bent that the data can easily be retrieved by the user. 
This is particularly true in instances where telephone 
numbers are to be referenced from the display screen. 
A need therefore arises for a user to be able to efficiently 
extract and use relevant information from either the re- 
ceived messages or previously stored data directly from 
the display screen. 

SUMMARY OF THE INVENTION 

In a first aspect, the present invention provides, in a 
computer system having a display screen responsive to 
contact thereto, a method of marking information dis- 
played on said display screen for selective applications 
comprising the steps of: (a) sensing contact to said 
screen to establish a start position; (b) sensing said con- 
tact being maintained at said start position for a prede- 



termined period of time to initiate a mark mode; (c) mark- 
ing the information displayed on said screen over which 
said contact traverses from said start position after initi- 
ation of said mark mode; (d) terminating said mark mode 

5 when said contact is removed from said screen. 

In a second aspect, the invention provides a com- 
puter system comprising: a touch sensitive display 
screen: means for responding to contact with a first lo- 
cation on said screen; means for initiating a mark mode 

10 when said contact with said first location on said screen 
exceeds a predetermined period of time; means for 
marking the data displayed on the areas of said screen 
over which said contact is moved from said first location 
after initiation of said mark mode, said mark mode being 

is terminated when said contact is removed from said 
screen. 

According to a preferred embodiment of the inven- 
tion, to be able to efficiently retrieve particular data being 
displayed, a user can mark the number or text that he 

20 desires by initiating a mark mode on the touch overlay 
membrane of the display. To initiate the mark mode, the 
user has to touch the screen so as to bring the cursor 
directly under the point of contact. Thereafter, the cursor 
becomes responsive to the touch on the display. If the 

25 user at this time touches the screen and then slides his 
finger (stylus), the cursor will follow the movement of the 
sliding contact point. To mark text (which includes letters, 
numbers and spaces), the user needs.to point to the be- 
ginning of the text that he wants to mark. Once the cursor 

30 is at the start of the desired text, the user would continue 
to touch the same position for a short period of time 
(which for this invention may for example be approxi- 
mately 0.5 to 1 .0 second). At the end of which time the 
system changes from a cursor sliding operation mode to 

35 a cursor mark mode. The device then sounds a beep to 
indicate that it has entered the mark mode. 

Upon initiation of the mark mode, text over which the 
finger moves is marked and a character count thereof is 
stored. When the user reaches the end of the text he 

40 wishes to mark, he releases his contact point with the 
display. At this time a beep having a second tone that is 
different from the first beep sounds from the device to 
indicate to the user that the marked text is accepted (i.e. 
the termination of the mark mode). A menu of operations, 

45 represented by either function keys or icons, now ap- 
pears on the display. At or about the same time, the 
marked text is displayed in a pop-up window to enable 
the user to verify that the desired text has been marked 
correctly. Upon confirmation of the marked text, the user 

so can press any one of the function keys or icons to begin 
an application that utilizes the marked text. When an ac- 
tivated application is the call application and the marked 
text is a telephone number, a call is made to the tele- 
phone number through the communications interface in- 

55 corporated within the device. 

The present invention thus provides a user with an 
efficient method of marking desired text on a touch sen- 
sitive screen. 
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The present invention preferably provides a user 
with the ability to retrieve displayed information for use 
in any one of a number of applications. The present in- 
vention preferably also provides a user with the ability of 
marking text from received messages and previously 
stored data. 

The present invention will now be described in more 
detail by way of example with reference to the accompa- 
nying drawings in which: 

Figure 1 is a perspective view of a personal commu- 
nications device according to the invention; 

Figure 2 is a plan view of the Figure 1 device; 

Figure 3 is an enlarged view of one of the available 
display screens of the Figure 1 device; 

Figure 4 is another example display screen of the 
Figure 1 device; 

Figure 5 is a simplified block diagram of the different 
components of the Figure 2 device which are rele- 
vant to the instant invention; 

Figure 6 is a broad overview, via different display 
screens : illustrating an embodiment of the instant 
invention; 

Figures 7A-7C illustrate the process of marking 
desired text on a display screen; and 

Figure 8 is a flow chart identifying the different 
processing steps of the instant invention method. 

DETAILED DESCRIPTION OF THE INVENTION 

With reference to Fig. 3, the display 10 of the per- 
sonal communications device of the instant invention 
can be selected to have a phone screen, by the user 
pressing the phone icon 26 at the bottom of the screen, 
which enables the user to use device 2 as a cellular tel- 
ephone. A more detailed discussion of the touch sensi- 
tive display is given in patent applications entitled "Per- 
sonal Communicator Having Improved Contrast Control 
For A Liquid Crystal Touch Sensitive Display" and "Per- 
sonal Communicator Having Improved Zoom and Pan 
Functions for Editing Information on Touch Sensitive Dis- 
play": both assigned to the same assignee of the instant 
invention and filed on November 1 , 1 993. The respective 
disclosures of those applications are incorporated herein 
by reference. 

When icon 28 is touched at the bottom of the screen, 
the mobile office screen shown in Fig. 4 is displayed. As 
can be seen, there are a number of different applications 
which the data processing system of device 2 is capable 
of. For example, device 2 has its own calendar and note 
pad, as well as a filer and an address book. In addition, 



it is capable of E-mail operation via the icon entitled 
"Mail" and a facsimile operation via the icon entitled 
"Fax". 

To enable the device to perform the variety of com- 

s munications, different communications links besides the 
radio link provided by the cellular telephone are also in- 
tegrated to device 2. These different types of communi- 
cation links are described in the above referenced 
co-pending application. 

10 A more detailed discussion of the integrated com- 
munications interface subsystem incorporated within de- 
vice 2 is given in part, or all, in the following co-pending 
applications: "Integrated FAX Send/Receive on Cellular 
and PSTN" (IBM Docket No. BC9-94-099), "Consistent 

75 User Interface for Cellular and PSTN Environments" 
(BC9-94-097), and "Modem Ring Indicate Signal From 
Either PSTN or Cellular System" (BC89-94-101). All of 
the just noted applications are assigned to the same as- 
signee as the instant invention and their respective dis- 

20 closures are incorporated by reference to the instant dis- 
closure. 

Fig. 5 is a simplified block diagram of device 2 of the 
instant invention which illustrates components only rele- 
vant to the instant invention. (A more detailed illustration 

25 of the different components of device 2 may be gleaned 
from the first referenced so-pending application). As 
shown, connect to the system bus 30 of the instant in- 
vention system is a processing unit 32. A number of dif- 
ferent types of memories such as RAM 34 : EPROM 36 

30 and additional memory 38 are also connected to system 
bus 30. Memory 38 may be in the form of a PCMCIA 
(personal computer memory card international associa- 
tion) standard card inserted to receptacle. slot 12 of de- 
vice 2. See Fig. 1 . Memory 38 may include extra RAM 

35 or ROM for storing additional data to be used in the sys- 
tem. EPROM 36 has stored therein a number of appli- 
cations among which are the ones mentioned and refer- 
enced as icons in the mobile office screen of Fig. 4. RAM 
34 is the working memory for CPU 32. 

40 Also connected to system bus 30 and being control- 
led by CPU 32 is a touch screen controller 40 whose 
function is influenced by a counter 42, which may actu- 
ally be the system clock. Being controlled by touch 
screen controller 40 is a tone generator 44. Further con- 

45 nected to touch screen controller 40 is a screen touch 
overlay 46 (referenced in the aforenoted applications 
filed on November 1 , 1 993) that senses pressurized con- 
tact, i.e. contacts made on display 10. A LCD display 10, 
superimposed by touch overlay 46 : is also connected to 

so bus 30 to be controlled by CPU 32. 

FurtheV connected to system bus 30 is an integrated 
cellular and PSTN interface 50 which, for the discussion 
of this invention, comprises a cellular link as exemplified 
by antenna 52 : a conventional type of serial port, con- 

55 ventional telephone jacks and a modem for data/fax 
communication. A more detailed discussion of the vari- 
ous features of the communications subsystem of device 
2 is given the first referenced co-pending application. 
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The operation of the instant invention system is dis- 
cussed with reference to Fig. 6, which shows an exem- 
plar E-mail message received by the system. As shown, 
screen 6A of Fig. 6 displays the main menu representing 
the "Mail" icon shown in the mobil office screen of Fig. 
4. Note that this screen is retrieved from system memory 
EPROM 36 and provides the user the option of viewing 
three different types of messages, namely the "Received 
Messages", the "Ready to Send" messages and the 
"Saved Messages". Screen 6B shows the three E-mail 
messages that have been received by device 2. To re- 
trieve screen 6B from screen 6 A : the function key entitled 
"Received Messages 11 was pressed by the user. 

When the user wants to view a particular message, 
he presses the block that outlines the message. For ex- 
ample, when the user presses the block with heading 
"Joe Smith", the message relating to Joe Smith is dis- 
played, such as for example in screen 6C. As shown, 
screen 6C is divided into two areas, namely a text area 
62 and a function area 64. Text which includes numbers, 
letters and spaces are displayed in area 62, whereas a 
number of function keys and icons are displayed in area 
64., Thus, by touching the back arrow icon, the display of 
device 2 would revert from screen 6C to screen 6B. In 
the same vein, if the telephone icon is pressed, the 
phone screen illustrated in Fig. 3 appears. Likewise, if 
the mobile office icon (shown to include an envelope in 
front of a pad) is pressed, the mobile office screen shown 
in Fig. 4 would appear. The question mark icon, if 
pressed, provides for queries. The shown function and 
the arrow keys are believed to be self-explanatory. 

Because of the communications nature of device 2, 
to be able to mark a particular portion of the information 
displayed on the text area would be useful. This ability 
to mark text is of particular import for E-mail and other 
types of written notes such as those in the "Note Pad". 
It is moreover desirable to be able to key in particular text 
from messages without having to jot it down on paper. 
Prior to the instant invention, if a user wants to dial a 
phone number displayed on a screen, he would first need 
to jot down the number. Thereafter he has to return to 
the phone screen. It was only after then would the user 
be able to dial the number. This was found to be ineffi- 
cient. 

For the instant invention, instead of having to copy 
the desired text, the user can directly mark the text on 
the screen for further application. The inventive method 
is discussed with reference to Fig. 7a-7c. 

As shown in Fig. 7a : appended to the bottom of Joe 
Smiths' staff meeting minutes are the telephone num- 
bers for both Joe Smith and Bob Jones, the person who 
gave the treasury report highlights. If the user wants to 
find out additional information in regard to the staff meet- 
ing, he would naturally want to call up either Joe Smith 
or Bob Jones. Suppose he wants to talk to Joe Smith. 
He can easily accomplish this by first positioning the cur- 
sor 66 on the screen to the start position of the text that 
he wants to mark, which in this instance is the phone 



number of Joe Smith. This he does by touching the 
screen and moving his finger (or stylus) to the first 
number of Joe Smiths' telephone number, as at this time 
the system is in the cursor sliding mode and is pro- 

s grammed to follow the contact point on the screen. As 
shown in Fig. 7A : cursor 66 is positioned underneath the 
first "5" of Joe Smiths' telephone number. At this time, 
the system remains in the cursor sliding mode, that is, 
the cursor is responsive to and continues to follow the 

10 movement of the user's finger. 

Because the user wants to mark the telephone 
number for later use, he needs to initiate the "marking 
mode" of the system. This he does by holding his finger 
at the same location of the screen for a predetermined 

is period of time. For the instant invention embodiment, this 
period of time is taken to be approximately 0.5 to 1 .0 sec- 
ond. In the system, this time is provided by counter 42 
(or the system clock) of Fig. 5. In other words, counter 
42 acts as a timer which reports to touch screen control- 

20 ier 40, which in turn is receiving from the sensor overlay 
46 an input that the screen is being touched by the user. 

Once it has been determined that the user's finger 
has been held still at the same location of the screen for 
the requisite amount of time, screen controller 40 would 

25 activate tone generator 44 to sound a beep, which tells 
the user that the system has now entered into the mark- 
ing mode. 

Upon hearing the marking mode beep, the user can 
move his finger on the screen to mark the desired text. 

30 it should be appreciated that contact should continuously 
be made by the user to the screen after the initiation of 
the mark mode in order for the mark mode to continue. 
Thus, as shown in Fig. 7B, the user has moved cursor 
66 to the last number "4" of Joe Smiths' telephone 

35 number. As cursor 66 is moved along the desired text, 
the text over which the contact point is moved, or over 
which cursor 66 traverses, are accentuated, either high- 
lighted or set in reverse video, as indicated by the dotted 
box outline 68. 

40 The marking mode continues as long as the user 
maintains his finger on the screen. Thus, the marked text 
can vary anywhere from one character to the entire text 
area 62. The marking mode is terminated when the user 
removes his finger from the screen. The end of the mark- 

45 jng mode is signified by tone generator 44 generating a 
second beep that is different from the beep that signifies 
the initiation of the mark mode. 

At the end of the marking mode the marked text is 
displayed in a pop-up window 70 as shown in Fig. 7C. 

50 The pop-up screen provides the user with a confirmation 
that the correct text has been marked. In addition to 
pop-up screen 70, a number of function keys also appear 
on the screen. The different function keys each provide 
an option to be taken with respect to the marked text. For 

55 example, if the user determines that the correct text has 
been marked, as for example the telephone number of 
Joe Smith, he can next instruct the system to dial that 
number by pressing the "Dial" function key 72. Alterna- 
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tively, if he changes his mind, he can un-mark the text 
by touching function key 74. Furthermore, he can cancel 
the marked text by pressing the back arrow icon. Thus, 
for the instant invention, without having to jot down any- 
thing, the user can directly call someone by simply mark- 
ing the desired telephone number and then pressing the 
"Dial" function key. 

In addition to marking telephone numbers, the 
present invention also is capable of marking other text. 
For example, if the user wants to find out the number of 
times Bob Jones has appeared on the text, he only needs 
to mark "Bob Jones" and then press the "Find" function 
key 76. The "Find" function can be extended to the var- 
ious messages stored in the "Mail" directory as well as 
other files kept at the different directories, such as the 
"Note Pad" and the "Address Book" directories. 

Even though the dialling of a phone number is dis- 
cussed with reference to the marking of a telephone 
number, in a co-pending application entitled "Apparatus 
and Method Therefor of Intelligently Searching for Infor- 
mation in a Personal Communications Device" 
(BC8-94-0282), assigned to the same assignee of the 
instant invention, the marked text does not have to be 
the telephone number. Briefly, in the just noted co-pend- 
ing application, by marking an appropriate abbreviated 
search term, the telephone number of Joe Smith may be 
directly dialled without any direct reference to his tele- 
phone number. Moreover, the marked text may be used 
in a number of applications aside from the application in 
which the text is being viewed and marked. For example, 
the marked telephone number illustrated in Figs. 7A-7C 
may be inserted to the address book, and particularly into 
the Joe Smith address. Or, for that matter, the marked 
text may be routed to the "To Do" directory file for future 
calling of Joe Smith, if the user deems it more appropri- 
ate to call Joe Smith at a later time. 

In any event, the marked number is sent to the inte- 
grated cellular and PSTN communications interface 50 
to dial the call number. 

The operation of marking the text and then dialling 
it is given in the flow chart of Fig. 8. As shown, the oper- 
ation starts with the scan screen service routine, which 
is one of the routines stored in EPROM 36 of the system, 
and is used by most, if not all, of the applications that run 
on the device. Putting it simply, scan screen service mon- 
itors the buttons (or the function keys) and the icons dis- 
played on area 64 and the text displayed on area 62 of 
the screen. Whether or not one of the function keys or 
icons is pressed is determined in button press routine 
802. Whether the text area, such as 62 in Fig. 6, is 
pressed is determined in block 804. If the text area is 
pressed, the cursor is moved to the pressed position per 
block 806A. At the same time, a timer, which may be 
counter 42 or the system clock, is set at block 806B. 
Thereafter, a further determination is made in block 808 
on whether or not the text area is pressed. If not, the proc- 
ess returns to block 802 to determine whether any of the 
function keys and/or icons have been pressed. 



If indeed the text area is pressed, a determination is 
made on whether there is a new cursor position per block 
810. If there is a new cursor position, the processor de- 
termines the system as remaining in the cursor sliding 

s mode and the process returns to blocks 806a and 806b 
to move the cursor to a new position and reset the timer. 
If it is determined that the cursor position has not been 
moved, i.e. the user has kept his finger on the same lo- 
cation of the screen, then a determination is made by the 

10 processor per block 812 on whether or not a predeter- 
mined time period has elapsed. If it has not, the system 
would again determine whether or not the text area con- 
tinues to be touched per block 808 and whether the user 
has moved his finger per block 810. 

is if jt is determined that a predetermined period of time 
has passed with the cursor being maintained at the same 
location, the process proceeds to the marking mode by 
sounding a beep : generated by tone generator 44, per 
block 814. The system continues to scan the touch sen- 

20 sitive display to determine whether the text area contin- 
ues to be touched in block 816. If it is, the area of the 
screen from the point where the marking mode begins 
to the contact point is marked, by either accentuating the 
marked text or by reverse video of the same, in block 

25 818. This process continues until the system senses that 
the text area no longer is being touched, i.e. the user 
ceases pressing the display screen. With the removal of 
the contact, a second beep is provided by tone generator 
44 per block 820, to notify the user that the mark mode 

30 is now terminated. 

At the same time, the pointer designating the begin- 
ning of the marked text and the number of character 
counts, i.e. the text that was marked, are provided to a 
storage such as RAM 34, for use in the different appli- 
es cations. 

As was mentioned previously, one of the applica- 
tions is the calling application which is illustrated in Fig. 
7C. With the beginning of the calling application, as the 
marked number is displayed in the pop-up window 70, 

40 yet another beep, which is different from the beeps out- 
put at blocks 814 and 820, is provided by tone generator 
44, per block 822a. The pop-up window 70 is deemed to 
be a confirmation in block 822b. A confirmation is next 
made by the system in block 824 on whether the marked 

45 text is correct. If it is, key 72 is touched per block 826 to 
call the confirmed telephone number. If not, the marked 
text can be unmarked by the user touching the unmarked 
button 74 per block 828. Thereafter, the system returns 
to the call scan screen service per block 830. 

50 

Claims 

1. In a computer system having a display screen 
55 responsive to contact thereto, a method of marking 
information displayed on said display screen for 
selective applications comprising the steps of: 
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(a) sensing contact to said screen to establish 
a start position: 

(b) sensing said contact being maintained at 
said start position for a predetermined period of s 
time to initiate a mark mode; 

(c) marking the information displayed on said 
screen over which said contact traverses from 
said start position after initiation of said mark 10 
mode; 

(d) terminating said mark mode when said con- 
tact is removed from said screen. 

15 

2. A method according to claim 1 , wherein one of said 
applications comprises a communications applica- 
tion and wherein, when said marked information is 
representative of a call number the method further 
comprises: 20 

sending said marked information to a commu- 
nications interface means incorporated in the sys- 
tem: which interface means enables communication 
with other communication systems, thereby to initi- 
ate a call to said call number. 25 

3. A method according to claim 2, wherein said call is 
initiated only after a step of confirming that said 
marked information is representative of a desired 
call number. 30 

4. A method according to claim 2 or claim 3, wherein 
the operation of sending marked information to a 
communication interface is initiated by a user touch- 
ing a predetermined area on said screen. 35 

5. A method according to any of claims 1 to 4, wherein 
information displayable on said screen includes text 
and icons; and 

wherein said step of marking displayed infor- 40 
mation over which the contact traverses includes the 
step of displaying said marked information as high- 
lighted or reverse video text. 

6. A method according to claim 5, further comprising 45 
the steps of: 

contacting one of said icons on said screen to 
initialize an application: 

utilizing said marked text in said application. 

50 

7. A method according to any one of the preceding 
claims, including: 

providing a tone when said mark mode is ini- 
tiated: 

providing a second tone when said mark mode 55 
is terminated. 

8. A computer system comprising: 



a touch sensitive display screen: 

means for responding to contact with a first 
location on said screen; 

means for initiating a mark mode when said 
contact with said first location on said screen 
exceeds a predetermined period of time; 

means for marking the data displayed on the 
areas of said screen over which said contact is 
moved from said first location after initiation of said \ 
mark mode : said mark mode being terminated when - f 
said contact is removed from said screen. 

h 

9. A computer system according to claim 8, further ' 
comprising: 

communications interfacing means; 

means for sending said marked data to said 
communications interfacing means for effecting 
communication to a call number corresponding to 
said marked data. 

10. A computer system according to claim 8 or claim 9, 
further comprising: 

means for generating at least a first tone when 
said mark mode is initiated and a second tone when 
said mark mode is terminated. 

11. A computer system according to any one of claims 
8 to 1 0, which is adapted to display said marked data 
as highlighted or reverse video text. 

12. A system according to any one of claims 8 to 11, 
wherein said communications interface means is 
capable of providing both cellular and PSTN com- 
munications. 

3. A system according to any one of claims 8 to 12 : 
wherein said display screen is divided into at least 
a first area where text is displayed and a second 
area where function keys or icons representative of 
different applications are displayed; and 

wherein, after text is marked in said first area, 
touching of a call function key or a call application 
icon in said second area activates said communica- 
tions interface means to call said call number. 
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